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Abstract —Research on information embedding, particularly 
information hiding techniques, has received considerable attention 
within the last years due to its potential application in multimedia 
security. Digital watermarking, which is an information hiding 
technique where the embedded information is robust against mali¬ 
cious or accidental attacks, might offer new possibilities to enforce 
the copyrights of multimedia data. In this paper, the specific case 
of information embedding into independent identically distributed 
(IID) data and attacks by additive white Gaussian noise (AWGN) 
is considered. The original data is not available to the decoder. For 
Gaussian data, in 1983, Costa proposed a scheme that theoretically 
achieves the capacity of this communication scenario. However, 
Costa’s scheme is not practical. Thus, several research groups 
have proposed suboptimal practical communication schemes 
based on Costa’s idea. The goal of this paper is to give a complete 
performance analysis of the scalar Costa scheme (SCS), which is 
a suboptimal technique using scalar embedding and reception 
functions. Information theoretic bounds and simulation results 
with state-of-the-art coding techniques are compared. Further, 
reception after amplitude scaling attacks and the invertibility of 
SCS embedding are investigated. 

Index Terms —Blind digital watermarking, communication with 
side-information, information embedding, scalar Costa scheme. 


I. Introduction 

R ESEARCH on information embedding has recently 
gained substantial attention. This is mainly due to the 
increased interest in digital watermarking technology that 
potentially can solve copyright infringements and data integrity 
disputes. Digital watermarking is considered to be the im¬ 
perceptible, robust, secure communication of information by 
embedding it in and retrieving it from other digital data. The 
basic idea is that the embedded information—the watermark 
message—travels with the multimedia data wherever the 
watermarked data goes. Recently, many different watermarking 
schemes for a large variety of data types have been developed. 
Most of the work considers still image data, but watermarking 
of audio and video data is popular as well. Theoretical limits 
of digital watermarking have been investigated since about 
1999 [1]—[3]. In general, watermark embedding techniques and 
attacks against watermarks have to be designed specifically for 
certain host data types. A particularly interesting case is that of 
independent identically distributed (IID) host data and attacks 
by additive white Gaussian noise (AWGN). The analysis of 
more complicated scenarios can often be ascribed to this special 
case [4]—[6]. 

Manuscript received February 4, 2002; revised November 20, 2002. This 
work was supported by the German Research Foundation (DFG) under Grants 
GI198/6-1, HU 634/4-2, and HU 634/4-3. The associate editor coordinating the 
review of this paper and approving it for publication was Dr. Ali N. Akansu. 

The authors are with the Lehrstuhl fur Nachrichtentechnik, Universitat Er- 
langen-Niimberg, Erlangen, Germany. 

Digital Object Identifier 10.1109/TSP.2003.809366 


In this paper, we focus on information embedding into IID 
host data facing AWGN attacks. Throughout the paper, we de¬ 
note the investigated scenario as a watermarking scenario. How¬ 
ever, it should be emphasized that AWGN is not the optimum 
attack against embedded watermarks for all types of host data. 
Thus, information embedding might be a more correct term than 
digital watermarking since the robustness requirement is weak¬ 
ened to robustness against AWGN. We further constrain the dis¬ 
cussion to blind reception, meaning that the decoder has no ac¬ 
cess to the original data. 

Fig. 1 depicts a block diagram of the considered blind water¬ 
marking scenario. A watermark message m is embedded into 
IID original data x of power o\ to produce the watermarked 
data s. The difference w = s — x is denoted the watermark 
signal. Here, we consider only embedding techniques giving a 
zero-mean watermark signal w with power limited to of. Next, 
AWGN v of power of, is added to the watermarked data s. 
This process, which is denoted AWGN attack, produces the at¬ 
tacked data r. The attacked data r is identical to the received 
data r, which is input to the watermark decoder. The embed¬ 
ding process and decoding process is dependent on the key K to 
achieve security of the communication. Usually, a key sequence 
k with the same length as x is derived from the key K. In this 
paper, x, w, s, r, and k are vectors of identical length L x , and 
x n , w n , s n , r n , and k n refer to their respective nth elements. 
Random variables are written in Sans Serif font, e.g., xfor 
a scalar random variable and x for a vector random variable. 

Watermark communication, as shown in Fig. 1, can be con¬ 
sidered as communication with side-information at the encoder. 
This has been first realized in 1999 by Chen and Wornell [7] 
and Cox et al. [8], Chen and Wornell introduced an important 
but almost forgotten paper by Costa into the watermarking com¬ 
munity. Costa [9] showed theoretically that the channel capacity 
for the communication scenario depicted in Fig. 1 with an IID 
Gaussian host signal x is 

Gics = ^log 2 (l +^f) (1) 

independent of of The suffix “ICS” stands for ideal Costa 
scheme and is used here to distinguish the theoretical per¬ 
formance limit from that of suboptimal schemes discussed 
below. The performance of ICS depends solely on the wa- 
termark-to-noise power ratio WNR = 101og 10 (cr^/cr^) (in 
decibels). The result (1) is surprising since it shows that the 
original data x need not be considered as interference at the 
decoder, although the decoder does not know x. Costa presents 
a theoretic scheme that involves a random codebook U L:,: , 
which is 

U Lx = {u ; = w* + onti 1 1 G {lj 2, ..., L u } 

w ~ Af(0, ofl L J, x ~ J\f( 0, o 2 x I l J} (2) 
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Fig. 1. Blind watermark communication facing an AWGN attack. 


where w and x are realizations of two L,.-dimensional inde¬ 
pendent random processes w and x with Gaussian probability 
density function (PDF). Ly is the total number of codebook en¬ 
tries, and Tj J:r denotes the L x -dimensional identity matrix. For 
secure watermarking, the codebook choice must be dependent 
on a key K. There exists at least one such codebook such that for 
L x —> oo, the capacity (1) is achieved. Note that the optimum 
choice of the parameter a depends on the WNR and is given by 

* _ 1 _ /o\ 

ICS ~~ ol + <j2 - 1 + 10 -WNR[dB]/10 • 

The ideal Costa scheme (ICS) is not practical due to the in¬ 
volved huge random codebook. Therefore, several suboptimal 
implementations of ICS have been proposed since 1999. A nat¬ 
ural simplification of ICS is the usage of a structured codebook 
U Lls , which in the most simple case can be constructed by a con¬ 
catenation of scalar uniform quantizers. This approach, which is 
constrained to a sample-wise (scalar) embedding and extraction 
rule, is denoted in this paper as the scalar Costa scheme (SCS). 
The accurate and complete performance analysis of SCS is the 
main topic of this paper. 

Before discussing the SCS, we give a brief review of related 
research on the implementation of Costa’s scheme. Chen and 
Wornell developed in 1998 quantization index modulation 
(QIM), which provides good performance for low channel noise 
but is not robust for channel conditions with [10], [1]. 

In 1999, they improved the QIM idea using Costa’s approach 
and named the new scheme QIM with distortion compensation 
(DC-QIM) [11], Most of the work of Chen and Wornell con¬ 
centrates on high-dimensional embedding techniques where 
the dimensionality tends to infinity. This approach enables 
the analytical derivation of performance bounds. However, 
little is said about the performance of currently implementable 
schemes. Further, simulation results using state-of-the-art 
channel coding techniques are not provided. Chen and Wornell 
also discuss a simplification of DC-QIM where the indexed 
quantizers are derived via dithered prototype quantizers. This 
technique is investigated particularly for the case of uniform 
scalar prototype quantizers, which is denoted as distortion 
compensated dither modulaton (DC-DM). Chen and Wornell 
present a coarse performance analysis of DC-DM that is based 
on minimum-distance arguments and the variances of the 
watermark and the attack noise. However, the specific shape of 
the involved PDFs of the transmitted and received signals are 
not modeled accurately so that tight performance limits cannot 
be computed. 

Ramkumar and Akansu [ 12]—[ 16] propose a blind water¬ 
marking technique based on periodic embedding and reception 
functions for self-noise suppression (host signal interference 


reduction). In particular, low-dimensional versions of this 
approach, e.g., with scalar embedding and reception functions, 
are closely related to suboptimal implementations of Costa’s 
scheme. Ramkumar and Akansu consider during their analysis 
a proper modeling of the PDFs of the transmitted signals. 
However, their analysis of the receiver performance involves 
approximations that are only valid if adjacent codebook entries 
for identical messages are far from each other. This assumption 
is not valid for a large range of practically relevant WNRs. Fur¬ 
ther, Ramkumar and Akansu present a capacity analysis based 
on an equivalent noise variance derived from the PDFs of the 
transmitted signal. This analysis is a good approximation only 
for low WNRs. For high WNRs, evaluation of the presented 
capacity formula results in values above the Shannon limit. 
Nevertheless, it should be emphasized that certain versions 
of the technique proposed by Ramkumar and Akansu show 
good performance particularly for very low WNRs. The SCS 
outperforms their approach in the range of typical WNRs only 
slightly [17], 

Chou et al. [18] exploit the duality of communication with 
side-information at the encoder to source coding with side-in- 
formation at the decoder to derive a watermarking scheme based 
on trellis-coded quantization. This work can be considered as 
an extension of the research on practical implementations of 
Costa’s scheme in the direction of high-dimensional embedding 
and reception rules. However, research in this direction is diffi¬ 
cult, and little progress has been made within the last few years. 
Up to now, performance results that are better than the theoret¬ 
ical capacity limit of ST-SCS propose (see Section V) have not 
been published. Latest results by Chou et al. [19] show at least 
a slight improvement of turbo coded trellis-based constructions 
over simple SCS communication using coded modulation tech¬ 
niques. Note also that SCS communication might still remain 
attractive due to its simplicity, even if superior performance of 
high-dimensional embedding techniques can be shown in fu¬ 
ture. 

Note also that principles of Costa’s work on communica¬ 
tion with side information have recently gained some attention 
within multiuser communications [20]-[22], 

The goal of this paper is to summarize theoretical and 
experimental results on the performance of the practical SCS 
embedding and reception technique. An accurate performance 
analysis is based on properly derived PDFs of the transmitted 
and received data. Further, a comparison of the performance of 
state-of-the art coding techniques with theoretical performance 
limits is given. In Section II, SCS is derived formally, and 
the encoding and decoding process is outlined. Theoretical 
performance limits of SCS are derived in Section III. Exper¬ 
imental results for SCS communication at high rates (low 
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noise power) are given in Section IV. Section V discusses SCS 
communication at low rates, which is particularly important for 
robust digital watermarking. Section VI discusses the important 
extension of the AWGN attack to an attack with additional 
amplitude scaling. An efficient algorithm for the estimation 
of such amplitude scaling attacks is presented. Finally, the 
invertibility of SCS watermark embedding is investigated in 
Section VII, which is of interest if the distortion introduced by 
watermark embedding should be reduced or even removed by 
the legal user of a watermarked document. 
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Fig. 2. SCS watermark embedding of the watermark letter d n , encrypted with 
key k n , into the original data element x n . 


II. Scalar Costa Scheme 

For a practical implementation of Costa’s scheme, the usage 
of a suboptimal, structured codebook is proposed, while leaving 
the main concept of Costa’s scheme unchanged. Besides being 
practical, the developed scheme is independent from the data 
distribution. This property can be achieved for a properly chosen 
embedding key sequence k [5], [6]. When no key is used, a rea¬ 
sonably smooth PDF p x (x) and cr 2 cr^, cr 2 must be assumed. 
To obtain a codebook with a simple structure, U Lx is chosen to 
be a product codebook of dithered uniform scalar quantizers, 
which is equivalent to an L x -dimensional cubic lattice [23]. 


A. SCS Encoder 

First, the watermark message m = b, where b is a binary 
representation of m, is encoded into a sequence of watermark 
letters d of length L x . The elements d n belong to a D-ary al¬ 
phabet V = {0, 1, ..., D — 1}. D-ary signaling denotes SCS 
watermarking with an alphabet V of size D = \D\. In many 
practical cases, binary SCS watermarking ( d n £ T> = {0, 1}) 
will be used. 

Second, the L x -dimensional codebook U Lx of Costa’s 
scheme is structured as a product codebook U Lx = 
U l o U 1 o • • • o U 1 of L x one-dimensional (1-D) compo¬ 
nent codebooks U 1 , where all component codebooks are 
identical. For D-ary signaling, the component codebook U 1 
must be separated into D distinct parts so that 

U 1 =D 0 1 UD 1 1 U---UDiu---UDl,_ 1 . (4) 

The codebook U 1 is chosen to be equivalent to the representa¬ 
tives of a scalar uniform quantizer with step size aA/D, which 
is formally denoted as 


U l = \ u = la A + — aA 


deV,l e 


(5) 


l enumerates all quantizer representatives of a prototype scalar 
quantizer with step size aA, and d introduces a shift of the pro¬ 
totype quantizer. The dth sub-codebook of U 1 is given by 


U\ = 


u = laA + — aA 



( 6 ) 


so that each sub-codebook is equivalent to the representatives of 
a scalar uniform quantizer with step size ctA. 

A simple and efficient encryption method for the SCS code¬ 
book is the derivation of a cryptographically secure pseudo¬ 


random sequence k from the watermark key K, with k n £ 
[0, 1), and the modification of each component codebook so that 

U 1 (k n ) = | u n = (l + k n )aA + aA 

(7) 

Without knowing k, it is practically impossible to reconstruct 
the codebook U Lx ( K ) used for watermark embedding. Note 
that the presented encryption does not modify any codebook 
properties being important for communication reliability, e.g., 
the distance between different codebook entries. 

For a Costa-type embedding of the watermark letters d, a 
jointly typical pair (uo, x) has to be found, which is equiva¬ 
lent to finding a sequence q = w/a = {vlq/ a) — x, which is 
nearly orthogonal to x. This search can also be considered to 
be a quantization of x with an L x -dimensional quantizer, where 
each quantizer representative is derived from the codebook en- 
triesu £ U Lx viau/ct. Wepropose ascheme in that this process 
is reduced to the sample-wise operation 

(In = Qa j^n-A + O j - f*n-A (j^ + k„X) 

' ' ' ( 8 ) 
where Q a {•} denotes scalar uniform quantization with step 
size A. Finally, the transmitted watermark sequence is given by 

w = Uo — ax = aq (9) 

and the watermarked data is 

s = x + w = x + aq. (10) 

A block diagram of the presented watermark embedding scheme 
is depicted in Fig. 2. Fig. 3 shows an example input-output char¬ 
acteristic for a = 0.6. The embedding of d n £ V can be 
expressed as subtractive dithered quantization, where A(k n + 
dn/D ) is the dither sequence, and A is the step size of the uni¬ 
form scalar quantizer. Note that the quantization error q and, 
thus, w as well, is almost orthogonal to the quantizer input x 
for an almost uniform original data PDF in the range of one 
quantization bin. For the given codebook encryption by a uni¬ 
formly distributed key sequence k, it can even be shown [24], 
[25] that q and w are statistically independent from x, as it is 
in Costa’s ideal scheme. Further, the power of the quantization 
error is always E {q 2 } = A 2 /12 for the given distribution of 
the key sequence. 





1006 


IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 51, NO. 4, APRIL 2003 



SCS embedding depends on two parameters: the quantizer 
step size A and the scale factor a. For a given watermark power 
er^, these parameters are related by 



o [4/ \/ \ 2 

A 


(ID 


ceived data r. This extraction process operates sample-wise, 
where the extraction rule for the nth element is 

Va = Qa {r n ~ K A} - (r n - k n A). (12) 

For binary SCS, \y n \ < A/2, where y n should be close to zero 
if d n = 0 was transmitted and close to ±A/2 for d n = 1. 
Second, depending on the type of error correction encoding of 
d, soft-input decoding algorithms, e.g., a Viterbi decoder for 
convolutional codes, can be used to decode from y the most 
likely transmitted watermark message m. 

C. QIM and Dither Modulation 

QIM, as described in [10] and [26], is a special case of Costa’s 
scheme, where a = 1, regardless of the WNR. As a result, QIM 
can achieve the capacity of ICS as the WNR tends to infinity. 
However, for > <r^, which is relevant in watermarking ap¬ 
plications, reliable communication is difficult since the quan¬ 
tizer cells are too small. Dither modulation (DM) with scalar 
prototype quantizers, as described in [10] and [26], relates to a 
general QIM scheme like SCS relates to an ideal Costa scheme. 
Note that throughout this paper, DM is always considered to op¬ 
erate with uniform scalar prototype quantizers. Then, DM can 
be considered a special case of SCS, where a = 1, regardless 
of the WNR. Since a is optimized in SCS for each WNR (see 
Section III-C), SCS can never perform worse than DM. In [11] 
and [27], Chen and Wornell discuss the extension of QIM using 
Costa’s ideas and denote the derived scheme as QIM with distor¬ 
tion compensation, which is basically Costa’s scheme described 
in a different way. 

III. Theoretical Performance Analysis 


Costa [9] determined aj CS , as defined in (3), to be the optimum 
value of a for the codebook (2). For a suboptimal codebook, 
e.g., the product codebook of scalar uniform quantizers used 
in SCS, the optimum value of a can be different. However, no 
analytical solution has yet been found. In Section III-C, the op¬ 
timum value for a in SCS, depending on the WNR, is computed 
numerically. 

B. SCS Decoder 

SCS decoding is very similar to the decoding process in ICS, 
except that the product codebook U Lx (K) = U 1 (kf) oU 1 (k\) o 
• • • o W 1 (% i _i), with W 1 (fc n ) as in (7), is used. Treating this 
codebook as a quantizer, the decoder acts as if it quantizes the 
received data r = x+w+v to the closest codebook entry. From 
this view of the decoding process, a sound interpretation of the 
encoding process results: The encoder perturbs the original data 
x by w to form the transmitted data s = x+w so that, with high 
probability, r will fall into the correctly indexed quantization 
bin. Simple hard-decision decoding of the nth watermark letter 
d n is achieved by scalar quantization of r n with U [ (k rl ). 

In general, the decoding reliability can be improved by de¬ 
coding an entire watermark letter sequence d, where the known 
encoding of m into d can be exploited to estimate the most likely 
d or, equivalently, to estimate the most likely watermark mes¬ 
sage rh. The simple codebook structure of SCS can be exploited 
to efficiently estimate d. First, data y is extracted from the re¬ 


The performance loss of SCS compared with ICS and a per¬ 
formance comparison of SCS and DM is of interest. Further, the 
optimization of the parameter a in SCS is desired. Performance 
is considered in terms of the watermark capacity of the specific 
schemes in case of an AWGN attack. The basis for an accu¬ 
rate performance evaluation are stochastic models for the water¬ 
marked data s and for the extracted data y. With these stochastic 
models, the capacities of SCS and DM in case of AWGN attacks 
are computed. The capacity computation for SCS involves the 
optimization of the parameter a. 

A. Distribution of Watermarked Data 

Due to the simple codebook structure in SCS and DM, the 
sample-wise embedding and extraction procedure, and the IID 
original data, s can be considered to be a realization of an IID 
stochastic process s with the PDF p $ (s). For performance eval¬ 
uation of the considered watermarking schemes, the conditional 
PDFs p s (.s|i7, k ) for all d CV are required. Conditioning on the 
key k is necessary since otherwise, the key hides any structure 
of the watermarked data. For simplicity, k = 0 is assumed for 
the presented illustrations. 

SCS and DM are based on uniform scalar quantization with 
step size A. It is assumed that the host data is almost uniformly 
distributed over the range of several quantizer bins. This 
assumption is reasonable in most watermarking applications, 
where the host-data power is much stronger than the watermark 
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Fig. 4. Qualitative diagram of the PDFs ps(s\d = 0, k = 0) (“—”) and 

p s (^|d = 1, k = 0) (“-”) of the watermarked data .s for binary DM and 

the SCS. 

power (tr* if). Note that the introduced assumptions may 
no longer be valid in case of SCS embedding for strong attacks 
since A might become quite large. For the following analysis, 
it is not necessary to accurately model the PDF p $ (s|c/, k) 
for all possible values of s. It is sufficient to have an accurate 
model for s in the range of several quantizer bins. Thus, for 
mathematical convenience, p $ (s\d, k) is considered periodic 
with period A. 

With the introduced assumptions, the shape of one period 
of p $ (,s|rf = 0, k), which is denoted by pg (s|d = 0, k), can be 
easily derived from the embedding rules for SCS and DM: 

DM: p~ s (s\d=0, k) =S(s) (13) 

SCS: re («|d = 0, k) = rect . (14) 

6 (•) denotes the Dirac impulse, and the rectangular signal is rect 
(a) = 1 for |a| < 0.5 and rect (a) = 0 for \a\ > 0.5. The PDFs 
p s (s|cZ 0, k) are almost identical to p s (s|o? = 0, k ), except 
for a shift by d/D ■ A, that is 

Ps ( s\d, k)=p s (^s- -^A d = 0, k"j (15) 

for both watermarking schemes. 

Fig. 4 depicts qualitatively the PDFs of the transmitted value 
s in case of binary signaling (d £ V = {0, 1}) for both con¬ 
sidered schemes. Note that p $ (s|c? = 0, k) and p s (s|d = 1, k) 
may even overlap for low WNRs and that the choice of A can be 
quite different for both schemes, which is not reflected in Fig. 4. 

B. Distribution of Extracted Received Data 

In this paper, attacks by AWGN independent from the original 
data and the watermark signal are considered. Thus, the PDF of 
the received data r is given by the convolution of the PDF of the 
transmitted data s and the PDF p v (v) of the additive channel 
noise: 

(16) 


Fig. 5. One period of the PDFs of the watermarked data s and the extracted 
data y for binary SCS (<r^ = 1, WNR = 2 dB, A = 5.7, and a = 0.61). The 
filled areas represent the probability of decision errors, assuming d = 0 was 
transmitted. 

Pr(r\k) = 7^7 ^ p r (r\d, k ) (17) 

' ' dev 

where denotes convolution. Equation (17) is valid for 
Prob(d) = 1/|X>|. Since it is assumed that p s (s|d, k ) is 
periodic with period A, p r (r\d, k) is also periodic with period 
A. One such period of p r (r\d, k ) is identical (except for a 
normalization factor) to the PDF p y (y\d, k ) of the extracted 
data y, where extraction with the correct key is assumed. A 
simple analytical expression forp y (y\d, k) is not known. Thus, 
p y (y\d, k) is computed numerically, as described in [5], [6], 
and [17]. Note that even for DM and a Gaussian PDF p v (v), 
the PDF of the received value will not be exactly Gaussian. 
Periodically overlapping Gaussian PDFs have to be considered 
due to the multiple representation of the watermark letters. 

The upper plot of Fig. 5 depicts one period of the PDF of the 
watermarked elements s conditioned on the transmitted water¬ 
mark letter d, and k = 0 for binary SCS. The lower plot shows 
the respective PDFs of the extracted received elements y after an 
AWGN attack. In case of using an incorrect key k at the receiver, 
the distribution of p y (y\d) will be uniform for any possible re¬ 
ceived signal. This is indicated by the dotted line in the lower 
plot of Fig. 5. 

C. Capacity Computation and Optimum SCS Step Size 

For the discrete memoryless channel, Gel’fand and Pinsker 
[28] and Heegard and El Gamal [29] showed that for communi¬ 
cation with side information at the encoder, the capacity is given 
by 

C — max (/(tv; r) — I(u;x)) (18) 

PU:W( U ’ H* y ) 

where the maximum is taken over all joint PDFs of the form 
p x (x) pu: w («,, w\x) p r (r\w, x ) and where u is an auxiliary 
random variable, and I(u: r) and /(tv; x) denote the mutual 
information between tv and the received data r and the mutual 


Pr(r\d, k) =p s (r\d, k)*p v (r ) 
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information between u and the side information (the original 
data) x, respectively. At the encoder, a specific realization of u 
is chosen, depending on the message m to be transmitted and 
the side information x available to the encoder. Appropriate 
realizations of u for all possible messages m and all possible 
side information x are listed in a codebook U, which must be 
known to the encoder and decoder. Equation (18) shows that the 
capacity of communication with side information at the encoder 
is given by the difference of information that the codebook U 
gives about the received data r and about the side information 
x. In general, maximization over all possible codebooks U 
and over all corresponding embedding functions is required. 
Here, the capacity of the suboptimum schemes DM and SCS is 
considered. SCS is constrained to a codebook based on scalar 
quantizers, which are parameterized by a and A, as shown 
in (5). a and A are related for fixed embedding distortion by 
(11). Thus, there is only one free codebook parameter for fixed 
embedding distortion so that the capacity of SCS is given by 



Fig. 6. Codebook parameter a. 


C'scs = max/(y; d). (19) 

a 


DM is a special case of SCS with a = 1 so that the capacity of 
DM is directly given by 


C*dm = I (y,d) 


a=l 


( 20 ) 


The watermark message is encoded such that for each data 
element, an alphabet T> = {0, 1, ..., D — 1} of watermark let¬ 
ters is used, where each letter is equiprobable. Then, the mutual 
information I(y, d ) is given by [30] 


I(y,d) = ~ J p y (y\k) log 2 p y (y\k) d y 

+ [Py(y\ d ) fc ) lo §2 Py(y\d, k) Ay. (21) 

It can be observed that J(y; d ) is completely determined by the 
PDFs p y (y\k ) and p y (y\d, k ), as derived in Section III-B for 
the case of AWGN attacks. 

For SCS, it is not possible to compute the maximization over 
a in (19) analytically since p y (y\k) and p y (y\d, k ) are given 
numerically. Thus, a is optimized numerically for WNRs in the 
range of —20 to 20 dB. The resulting values for a are shown in 
Fig. 6, and corresponding values for A are shown in Fig. 7. An 
approximative analytical expression for the optimum value of a 
has been derived experimentally, which is 


tt'SCS. approx — 

This leads, with (11), to 

Ascs, approx — ^12 (<72+2.710. (23) 

Fig. 6 and 7 also show the optimum value af cs derived by Costa 
for ICS [see (3)] and the corresponding value for A when using 
a ics i* 1 SCS. It can be observed that af cs is almost identical 
to the optimum value ag CS for SCS in case of positive WNRs. 


O +2.71<j2 • 


( 22 ) 



Fig. 7. Codebook parameter A for rrf = 1. 

However, for negative WNRs, af cs is too small for SCS. In this 
case, o:scs. approx defined in (22) is a better approximation for 
the optimum value of a for SCS. 

D. Capacity of SCS Watermarking Facing AWGN Attacks 

Fig. 8 compares the capacities of ICS, binary SCS, binary 
DM, and blind spread-spectrum (SS) watermarking for AWGN 
attacks. SCS watermarking does not achieve capacity but is not 
too far from an ideal scheme either. DM performs poorly for 
negative WNRs, where the optimum value of a is significantly 
smaller than 1. 

The term SS watermarking has been established in the water¬ 
marking community for watermark embedding by the addition 
of a statistically independent pseudo-noise signal w with power 
<7+ which is derived from the watermark message m and the key 
K. SS watermarking is one of the first methods used for water¬ 
marking (e.g., [31], [32]) and is still the most popular one. For 
a Gaussian original signal, AWGN attack, and for a Gaussian 
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Fig. 8. Capacity of blind watermarking facing an AWGN attack compared for 
ICS, binary SCS, binary DM, and blind SS watermarking. 

watermark signal w ~ Af( 0, cr^), the capacity of blind SS wa¬ 
termarking is given by the capacity of an AWGN channel [30], 
that is 

Css=1 2 ,o ^( 1+ ^fh)- m 

Note that cr~ >> and cr% within common water¬ 

marking scenarios. Thus, the performance of blind SS water¬ 
marking facing an AWGN attack is mainly determined by the 
document-to-watermark power ratio DWR = 101og 10 (<j^/cr^) 
(in decibels). This shows that blind watermark reception suffers 
significantly from original signal interference. The depicted ca¬ 
pacity of blind SS watermarking is for DWR = 15dB.Forweak 
to moderately strong attacks (i.e., WNRs greater than about —10 
dB) SCS watermarking outperforms SS watermarking by far 
due to the data-independent nature of SCS watermarking. How¬ 
ever, Fig. 8 also reveals that for very strong attacks (WNR < 
—15 dB), blind SS is more appropriate than SCS watermarking 
since here, the attack distortion dominates possible interference 
from the original signal. Note that ICS outperforms blind SS 
watermarking for all WNRs. 

Fig. 8 shows also that the binary SCS capacity is limited 
for high WNRs due to the binary alphabet V of watermark 
letters. Increasing the size D of the signaling alphabet T> en¬ 
ables higher capacities for high WNRs, as shown in Fig. 9. It 
can be observed that for very large signaling alphabets, the ca¬ 
pacity of SCS slowly approaches the capacity of ICS, or equiv¬ 
alently, Cscs(WNR) is slightly steeper than Dies (WNR) for 
high WNRs. 

IV. High-Rate SCS Communication 

High-rate SCS communication is of interest for scenarios 
with low attack noise, for instance, if information embedding 
into analog channels is desired [27], [33], [34]. Here, infor¬ 
mation embedding at rates R > 0.5 bit/element is considered 
high-rate watermarking since for these rates, the capacity of 
binary SCS is significantly lower than for D-ary signaling with 



WNR [dB] 


Fig. 9. Capacity of ICS and D-ary SCS watermarking facing an AWGN attack. 

D > 2, as shown in Fig. 9. It can be observed that the size 
of the alphabet T> has a significant influence only for WNRs 
larger than about 4 dB or, equivalently, R > 0.6 bit/element. 

Coded modulation techniques are used to combine D-ary sig¬ 
naling with binary error-correction coding. Here, the perfor¬ 
mance of SCS at R = 1 bit/element is investigated for different 
coded modulation techniques. As shown in Fig. 9, for R = 1 
bit/element, 3-ary signaling is as good as D-ary signaling with 
D > 3. Examples for 4-ary and 8-ary signaling are discussed 
here due to their efficient combination with binary coding tech¬ 
niques. We use 4-ary signaling for the classical trellis-coded 
modulation with convolutional codes (CC-TCM) as proposed by 
Ungerboeck [35]. We use 8-ary signaling in combination with a 
new trellis-coded modulation scheme with serial concatenated 
codes and iterative decoding (SC-TCM). A detailed discussion 
of coded modulation is beyond the scope of this work. More de¬ 
tails on these specific coded modulation schemes are given in 
[4] and [5]. The main goal is to demonstrate that with R = 1 
bit/element, low bit-error rates (BER < 10 -5 ) can be achieved 
within 1.6 dB of the capacity C'scs- 

Bit-error rates (BERs) around 10 -5 are achieved by CC-TCM 
and SC-TCM for WNR > 9.3 dB. The best performance for 
BER < 10 -5 was achieved by SC-TCM, with a minimum 
required WNR « 8.3 dB. However, note that the computa¬ 
tional complexity of ST-TCM and the codeword length (10 000 
information bits) is also quite large compared with CC-TCM. 
Fig. 10 compares the measured minimum WNR for achieving 
BER < 10 -5 with SC-TCM and CC-TCM with the capacity 
of SCS and ICS. Ideally, SCS with R = 1 bit/element is pos¬ 
sible for WNR > 6.7 dB. Thus, the discussed coded modulation 
schemes come within 1.6-2.6 dB of an optimal coding scheme 
for SCS. The distance to ICS is about 3.5 dB. 

V. Low-Rate SCS Communication 

In most watermarking applications, the attack distortion can 
be at least as large as the watermark embedding distortion. For 
the case of AWGN attacks, this means that a WNR of about 0 dB 
or less must be considered. For these distortion levels, reliable 
watermark communication can be achieved only at low rates. 
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Fig. 10. SCS watermark capacity compared with measured results (BER < 
10 _s ) using trellis-coded modulation. 

Binary SCS is sufficient for low-rate watermarking. Thus, the 
watermark message m, which is represented by a binary se¬ 
quence b, has to be encoded into a sequence b c = d of binary 
watermark letters d„ £ {0, 1}. In order to achieve communi¬ 
cation with low error rates, each bit of b has to be embedded 
redundantly into the original data x. Different methods for the 
redundant embedding of b are investigated below, and their per¬ 
formance for an AWGN attack is compared. 

A. Repetition Coding and Spread Transform 

The simplest approach for the redundant embedding of the 
information bits b into the original data x is the repeated em¬ 
bedding of each bit. An alternative approach for redundant em¬ 
bedding of the information bits b into the original data x is the 
spread-transform (ST) technique, as proposed in [7]. We found 
that repetition coding with SCS performs worse than ST with 
SCS (ST-SCS), which is not obvious at the first glance. Here, 
we illustrate the reason for this result. 

Let p denote the repetition factor for SCS with repetition 
coding, e.g., one information bit is embedded into p consecu¬ 
tive data elements. However, instead of deciding for each ex¬ 
tracted value y n what transmitted watermark letter d n is most 
likely, the decoder can directly estimate the most likely trans¬ 
mitted watermark information bit from p consecutive extracted 
values y n [4], [5], 

Spread transform watermarking has been proposed by Chen 
and Wornell [7]. A detailed description of this technique can 
be found in [4], [5], and [7]. Here, we focus on the general 
principle. In ST watermarking, the watermark is not directly 
embedded into the original signal x but into the projection 
x ST of x onto a random sequence t. Note that the term 
“transform,” as introduced by Chen and Wornell, is somewhat 
misleading since ST watermarking is mainly a pseudo-random 
selection of a signal component x ST to be watermarked. All 
signal components orthogonal to the spreading vector t remain 
unmodified. Let r denote the spreading factor, meaning the 
number of consecutive original data elements x n belonging to 
one element xfj. 
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WNR. [dB] 

Fig. 11. BER for SCS with repetition coding and ST-SCS watermarking. For 
identical watermarking rates (r = p), ST-SCS yields lower error rates than 
SCS with repetition coding. 

For watermark detection, the received data r is projected 
onto t as well. The basic idea behind ST watermarking is that 
any component of the channel noise v that is orthogonal to the 
spreading vector t does not impair watermark detection. Thus, 
an attacker, not knowing the exact spreading direction t, has to 
introduce much larger distortions to impair a ST watermark as 
strong as a watermark embedded directly into x. For an AWGN 
attack, the effective WNR r after ST with spreading factor r is 
given by 

WNR r = WNR! + 10 log 10 r. (25) 

Thus, doubling the spreading length r gives an additional power 
advantage of 3 dB for the watermark in the ST domain. How¬ 
ever, note that repetition coding and ST with p > 1 and r > 1, 
respectively, achieve more robustness against attack noise at 
the cost of a reduced watermark rate. For a fair comparison of 
SCS with repetition coding and ST-SCS, the watermark rate of 
both schemes should be equal, i.e., the repetition factor and the 
spreading factor should be equal (p = r). 

The BERs for SCS with repetition coding and ST-SCS 
after an AWGN attack have been measured for different 
WNRs. Fig. 11 shows simulation results for p = 2, 4, 8 
and r = 2, 4, 8. It has been observed that ST-SCS yields 
significantly lower BERs than SCS with repetition coding at the 
same watermarking rate. The predicted WNR gain of 3 dB for 
the same decoding reliability by doubling r can be observed. 
However, the WNR gain for SCS with repetition coding is less 
than 3 dB when p = 2. The observed effect can be explained 
by examining the specific structure of the codebook U in SCS. 

The multiple representations of a single watermark letter d n 
by several points in the signaling space lead to many nearest 
neighbors, which can lead to decoding errors. Fig. 12 shows a 
section of the two-dimensional (2-D) PDFs of pairs of received 
data elements r n in the case of an information bit b n > = 0 for 
SCS with repetition coding with p = 2; bright areas indicate 
high probabilities. The key sequence k has been set to zero for il¬ 
lustration purposes. The circles and crosses depict the codebook 










































EGGERS et al .: SCALAR COSTA SCHEME FOR INFORMATION EMBEDDING 


1011 



n. 

Fig. 12. Reception statistics for SCS with repetition coding with p — 2. 



Fig. 13. Reception statistics for ST-SCS with r = 2. 

entries corresponding to a transmitted watermark bit b n > = 0 
and b n ' = 1, respectively. Each circle is surrounded by four 
nearby crosses. Fig. 13 shows the corresponding 2-D PDFs in 
the case of ST-SCS with r = 2, where the spreading direction 
t was chosen to be the main diagonal. Obviously, any noise that 
is orthogonal to t does not affect the decision as to whether the 
transmitted bit was 0 or 1. Further, each circle is surrounded 
only by two crosses. Thus, the probability that AWGN pushes 
watermarked data into the area where a decoding error occurs is 
lower for ST-SCS than for SCS with repetition coding. 

Please note that the advantage of ST-SCS over SCS with rep¬ 
etition coding is only possible if the spreading direction t is not 
known to an attacker. Otherwise, an attacker would place all the 
noise in the direction t and the WNR-advantage vanishes. Fur¬ 
ther, ST-SCS watermarking with large spreading factors r might 
be impractical since perfect synchronization of the complete 
spreading vector t is necessary. In contrast, decoding in the case 
of SCS with repetition coding is possible when only some of the 



WNR [dB] 

Fig. 14. Performance improvement by spread-transform watermarking. 


watermarked data elements are synchronized. Another potential 
problem with large spreading factors r is that the original-data 
power in the ST domain might become so low that the assump¬ 
tion that the original data is approximately uniformly distributed 
in the range of one quantizer cell no longer holds; this assump¬ 
tion is used in quantization-based watermarking schemes like 
SCS and DM. As a consequence, the power of the watermark 
can no longer be predicted by A 2 /12. However, this problem 
can be avoided by using a key sequence k (Section II) that acts 
as a dither sequence that ensures a quantization noise power of 
A 2 /12. 

B. Capacity of ST-Watermarking and Optimal Spreading 
Factor 

ST-SCS watermarking should be considered a different sub- 
optimal approach to implement a transmission scheme with side 
information at the encoder. Thus, the achievable rate of ST-SCS 
might be larger than that of SCS. Note that ST-SCS can never 
perform worse than SCS since SCS is a special case of ST-SCS 
with r = 1. The optimum choice of the spreading factor r for 
attacks of differing noise powers is investigated. 

Fet Cst, r (WNR) denote the capacity of a specific wa¬ 
termarking scheme combined with a spread transform with 
spreading factor r for an AWGN attack with given WNR. 
Cst. l (WNR) is the capacity of the respective scheme without 
ST. The performance of ST watermarking can be computed 
from that of the respective scheme without ST by 

pa, 

r 

with WNR r = WNR + 10 log 10 r. 

Applying the ST technique the capacity of SCS and DM wa¬ 
termarking can be improved for WNRs lower than a certain 
WNR[4], [5]. For WNRs > WNR cr ; t , the optimal spreading 
factor r is 1, i.e., ST does not provide an additional capacity 
gain. The capacity of the ideal scheme ICS can not be improved 
by ST at all. 

Fig. 14 shows the capacities of SCS, ST-SCS, DM, and 
ST-DM. Since the achievable rates for SCS and DM water¬ 
marking are computed numerically, the corresponding WNR cl .; t 
are also obtained numerically. It can be found that for SCS, 
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Fig. 15. Reliable coded binary (ST-)SCS compared with theoretical limits. 
The measured points indicate the minimum WNR for that a specific coding 
technique achieves BER « 10 -5 . 

WNR crit> scs ~ 0.01 dB and for DM WNR (Til DM « 5.81 
dB. Fig. 14 shows also that DM can be improved significantly 
for WNR < WNR cr ;t : DM, where for SCS only a minor gain 
is accessible. Note that ST-DM performs worse than simple 
SCS for most practical WNRs. In addition, there is a constant 
gain of about 1.8 dB for ST-SCS over ST-DM in the range of 
negative WNRs. 

C. SCS With State-of-the-Art Channel Coding 

Repetition coding is known to be very inefficient. 
State-of-the-art error correction codes, e.g., turbo codes 
[36], outperform repetition coding by far. Therefore, simula¬ 
tion results for SCS communication using turbo coding are 
presented in Fig. 15. 

Fig. 15 shows the minimum WNR m ; n for which coded SCS 
watermarking gives BER « 10 -5 . It can be observed that turbo- 
coded (TC) SCS performs close to the capacity of SCS water¬ 
marking. The coding results for the code rates R c = 1/2 and 
R c = 1/3 can be translated to lower watermark rates R via ST 
watermarking, which is indicated by straight lines. ST-SCS wa¬ 
termarking with a code rate R c = 1/3 turbo code seems to be 
a very good choice for low-rate watermarking if any desired ST 
length r is applicable. Fig. 15 also shows that turbo-coded SCS 
combined with repetition coding is less efficient than ST-SCS. 
Nevertheless, repetition coding might be useful in practice since 
it can be implemented in a very flexible way. Any received data 
element r n with embedded watermark bit b n ; increases the es¬ 
timation reliability for b n i , where for ST watermarking, all data 
elements r n required for the computation of the projection rfj 
must be available to the receiver. 

VI. SCS Step-Size Estimation 

A practically important extension of the simple AWGN 
channel model considered so far is a possible constant ampli¬ 
tude scaling and DC offset. Further, it might be of interest to 
adapt the SCS quantizer step size A to the characteristics of 
the original data. In both cases, a blind receiver is confronted 


with the difficulty of finding the proper quantizer step size for 
SCS reception. There exist several approaches to combat this 
problem. The step size A could be related to some statistics 
of the original that can be estimated robustly at the watermark 
decoder. A brute force approach would be to search for the 
valid step size A by attempting to decode iteratively a valid 
message with different possible step sizes. Finally, we discuss 
here a specific step size estimation algorithm that is based 
on the analysis of histograms of received data r with SCS 
watermarks [5], [37]. 

Fig. 16 depicts an extension of Fig. 1, where the attacker 
scales the watermarked data s by g (usually g < 1) and in¬ 
troduces additive white Gaussian noise (AWGN) v, with v ~ 

ATr-offset, ol ), that is 

r = gs + v = g(x + w) + v. (27) 

Ideally, the receiver knows g and r 0 ff S et and thus compensates 
for the DC offset by subtracting r 0 flf se t and compensates for 
scaling by division by g (if g ^ 0). We characterize the at¬ 
tack strength by the effective watermark-to-noise power ratio 
WNR = 10 log w (g 2 al/a 2 v ) dB. 

At the receiver, after compensation for g and r 0 ffseti the ex¬ 
traction rule (12) can be applied to the signal r / . However, if no 
compensation for g and r 0 ff S et is applied, the proper codebook 
for SCS watermark reception is 

lAn, (I'll) — ^ U n — T T k n ^ oA r T /offset 

d n G £>, l E Z j . (28) 

Here, A,, = gA is the scaled quantizer step size that has to be 
used for SCS detection. 

We propose a technique for estimating the attack channel pa¬ 
rameters g and r 0 ff S et with the aid of a securely embedded pilot 
sequence d p ii 0 t = 0 of length . Security is achieved again 
by embedding the pilot dependent on a secure random key se¬ 
quence k, where k n G [0, 1). Note that estimation of A,. = gA 
is sufficient to enable SCS watermark reception, g can be de¬ 
rived when A is known to the receiver. 

The key idea behind our method for the estimation of A,, 
and roff S et is to analyze the 2-D histograms of the received 
samples r n , p ;i ot and the corresponding key values k n , where 
t pilot = (ro, pilot • • • pilot • • • r Lpilot - 1 , pilot) is the sequence 
of received samples with embedded pilot symbols d p ii 0 t = 0. 
The suffix “pilot” is suppressed subsequently since only pilot 
samples are considered in this subsection. 

Let p r .j < (r, k ) denote the 2-D PDF of the received signal sam¬ 
ples r n and the corresponding key values k n . Here, IID signals 
are considered so that the sample index n can be neglected in 
the statistical analysis. 

Fig. 17 shows examples forp r ^(r, k), where incorrect keys 
and correct keys are assumed in the upper and lower plot, respec¬ 
tively. Note that for illustration purposes, the step size A r in this 
example is relatively large compared with the host signal stan¬ 
dard deviation o x . Without knowing the correct k, no structure 
in the watermarked signal is visible. The received PDF basically 
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Fig. 16. Watermark communication facing an attack by amplitude scaling and AWGN with mean r 0 ff set . 




Fig. 17. Two-dimensional PDFs p ^ ( r, /. ). where the upper plot is valid for 
reception with an incorrect key, and the lower plot is valid for reception with the 
correct key. Bright areas indicate high probability. 


resembles the PDF of the host data which is a Gaussian distribu¬ 
tion in the given example. However, computing p r j < ( r , k ) with 
the correct key k reveals the inherent structure in the data with 
embedded pilot samples, p r .^ (r, k) shows stripes of high prob¬ 
ability. For fixed k, the distance between the peaks of two adja¬ 
cent stripes equals the step size A,, = gA (which gives A r = 10 
in the shown example). This structure appears since the rele¬ 
vant quantizer structure within SCS embedding is dithered by 
the product kA. 

In practice, p r .^(r, k ) has to be estimated via a 2-D his¬ 
togram, which requires proper discretization of the PDF vari¬ 
ables (r, k). We describe here an estimation algorithm based 
on Z/binfc different bins considered for the key value k. The 1-D 
conditional histograms of the samples r n with key k n G K v are 
analyzed separately, where 


— ik j-— < k < LAI 

-^binfc -L'hink 

for V G {0, 1, . . . , ibinfc 


1} and Lbinfe > 1- (29) 



Fig. 18. Total and conditional PDFs of the received pilot sequence. Lbinfc = 3 
different ranges for the key are distinguished. The example is for a Gaussian 
distribution of r n and for the parameters A r =10 and r 0 ff set = 0. 


is motivated by the observation that each PDF p r (r\k G IK,.) 
shows local maxima with a distance of A,,. Let p r (r) denote the 
PDF of the received signal samples r n . It can be assumed that 
p r ( r ) reflects more or less the host signal PDF (p r ( r ) ps p x (x )) 
if the embedding distortion and attack distortion is small rel¬ 
ative to the host signal power. An exact characterization of 
p r (r\k G IK,.) is not necessary for our purpose. A sufficiently 
accurate model is given by 

p r (r\k G K v ) 
ttp r (r\k G K v ) 


(30) 



= Pr ( r ) ^1 + 7 cos (^2irf 0 r - <f > 0 - 


27T 

bi no¬ 


where 7 is an appropriate constant with 0 < 7 < 1. The model 
parameters / (l and $0 are directly related to the unknown pa¬ 
rameters A,, and r 0 ffset- /o determines the distance between two 
local maxima, and <E>o determines their absolute position. The 
exact relationship is given by 

1 2ir 

fo — ^ and 'I'o = Ty Tpffeet = 27T_/'(T/' 0 ff' S el * (31) 


The Lbinfc conditional histograms will show local maxima with 
a relative distance of A. r . The absolute position of these maxima 
gives an estimate of r 0 ff se t- 

A. Parameter Estimation Based on Fourier Analysis 

We introduce a simple model for the conditional PDFs 
p r (r\k G K,,) of the received pilot elements in order to moti¬ 
vate the afterwards described estimation of /’offset ■ The model 


Fig. 18 depicts an example for the given model. The local 
maxima of the conditional PDFs p r (r\k G K v ) with a relative 
distance of A,. = 10 are clearly visible. 

The parameters fo and <Do of the model given in (30) have 
to be computed from the given conditional PDFs p r (r|fc G IK,,) 
and the given unconditional PDF p r (r). Fourier analysis is ap¬ 
propriate for this task since jo and $0 are the frequency and a 
constant phase contribution of the cosine term in (30). 
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For the vth conditional PDF, the normalized spectrum A v (/) 
is defined as 


A v {f)=F 


= T f pr(r\k £ Ky) _ 1 
1 Pr(r) 


= T < 7 cos ( 27r/or — 4>o — 


2 71" ( 1 

Lbin fc \ 2 


_|_ e -j(-4 > o-(27r/i b | llfc ){?;+(l/2)))g^ o _|_ j) 


■ (32) 


All Lbinfc spectra can be combined in an elegant way due to 
the systematically different phase at A v (fo) and A v (—fo). The 
Lbinfc spectra A v (f ) are multiplied by eK 27r / L ^A V prior to 
their summation to an overall spectrum A(f ), that is 


-^binfc 1 

A(f) = A v (f)^ (2n/Lbink)v 

v=0 

= ' yL ^ nk e -i^o+^/L hink )) s ( f Q _ ^ _ (33) 


Thus, for the model given in (30), |A(/)| has only one peak, 
which is located exactly at the frequency f ,. Further, <1> (I = 
— arg{A(/o)} — (7r/ibinfc)- Note that the multiplication by 
gK^/Lbinid^ j s superior to a multiplication by e>( 27r / ibinfc ) t ' f , 
which would correspond to a shift of the different conditional 
PDFs by v/Lkmk- In the latter case, the spectrum |H(/)| would 
have another peak at / = —/o, which increases the required 
sampling interval for the numerical computation of the condi¬ 
tional PDFs. 

The exact PDFs of the received signal do not fit exactly 
to the model given in (30). Further, in practice, the PDFs 
p r (r\k £ K v ) and p r (r ) can be only estimated from the L p ii ot 
pilot samples r. This estimation is obtained from histograms 
with Lbinr bins that cover the total range of all received 
samples. Based on these histograms, A v (f ) is computed at 
Ldft > Lbinr- discrete frequencies via a length-T oft DFT. 
Here, a single peak in the spectrum A(f) cannot be expected 
due to estimation errors and the inaccuracy of the model (30). 
Nevertheless, for T pl | ol sufficiently large, a dominating peak 
should occur at f,. Details of the outlined implementation are 
described in [5] and [37]. 


B. Estimation Performance for Different L p ;i ot 

The outlined algorithm for the estimation of A,, and r 0 fr S et is 
dependent on the following set of parameters: 

Lpilot length of pilot sequence; 

Lbinr : number of histogram bins used for the pilot sample 
value; 

Lbinfc :num her of histogram bins used for the key value; 
Ldft DFT length. 

The estimation accuracy also depends on the WNR and on the 
DWR. In this paper, the estimation performance for different 
pilot length L p ;i 0 t is discussed for WNR = —10 dB, ..., 5 dB. 
This range for WNR covers the most interesting range of attack 
strengths for where SCS watermarking might be useful. The 
DWR has been fixed to DWR = 20 dB, and the remaining 


parameters are Lbi m - = 50, Ldft = 1024, and Lbinfc = 5. 
Experimental results that support this choice of parameters are 
given in [5], 

The influence of the number L p ;i 0 t of received pilot elements 
is studied experimentally. For simplicity, g = 1, and no offset 
has been considered so that the estimator should ideally find 
A,. = A,, = A and f offset = 0. For the evaluation of the esti¬ 
mation performance, three different figures of merit have been 
used: 


relative error of A r : 


$A r 

relative error of /' 0 ff se i: 



8 r 


_ a/E {(foffset ~ 0) 2 } 

A r 


relative increase of bit-error probability: 

E {pb ~ Pb} 


3pb ~ 


Pb 


(34) 


(35) 


(36) 


(A,, and (5,. of f S et effectively measure the root of the mean squared 
estimation error relative to the exact step size A. These figures 
of merit have been chosen since not only is the variance of esti¬ 
mation errors important but a possible biased estimate is as well. 
The relative increase of the bit-error probability j>t, for uncoded 
binary SCS reception with estimated A,- and r 0 ff S et' s gi ven by 
8p b - It is sufficient to measure the expected difference of the 
bit-error probability since imperfect estimates A,, and r 0 ffset 
can only increase the bit-error probability on average, p^ of un¬ 
coded binary SCS is relatively high for the considered WNRs. 
However, many new parameters would have to be introduced 
for simulations with coded SCS communication, which would 
make a fair comparison more difficult. Further, the increase of 
Pb can be considered to be a good indicator for the effect of es¬ 
timation errors on coded communication. The free parameters 
can be optimized only for a certain range of different WNRs, 
where here, the focus is on WNR = —5 dB to WNR = 0 dB. In 
particular, the relative increase of the uncoded error probability 
6p b shows a local minimum for a certain WNR since for large 
negative WNRs, the estimation accuracy is decreased due to the 
strong noise, and for high WNRs, the absolute decoding error is 
so low that any decoding error increases the relative decoding 
error significantly. 

In general, it is desired to make the pilot sequence as short 
as possible; however, very short pilot sequences lead to an in¬ 
accurate PDF estimation and, thus, to incorrect estimations of 
A,- and r 0 ff S et- Fig- 19 shows the estimation performance for 
L p iiot = 250, 500, 1000, and 2000. Fig. 19(a) depicts 
which describes the relative estimation error of A,.. For L p ii ot = 
2000, <A, decreases monotonically with increasing WNR and 
is lower than 1% for WNR > —3 dB. Shorter pilot sequences 
lead to an increased relative estimation error. However, for some 
WNR, robust estimation is no longer possible at all. Lowering 
the WNR further introduces so much noise into the PDF estima¬ 
tion that the largest component of the computed DFT spectrum 
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Fig. 19. Estimation performance for different pilot lengths L p ii ot (DWR = 20 dB, Lbinr = 50, Lbink = 5). 


appears at any random frequency index 0 < l < £dft — 1. 
For Lpiiot = 250, this effect occurs for WNR < — 1 dB. For 
Lpiiot = 500, a minimum WNR of about —5 dB is required. 
Fig. 19(b) depicts S rhmet - which follows in general the behavior 
of Sa,. ■ The resulting relative increase of the uncoded error rate 
S Pb is shown with linear and logarithmic axes in Fig. 19(c) and 
(d), respectively. 6 Pb increases monotonically with decreasing 
pilot length L p ;iot. Further, it can be observed again that for 
some low WNRs, the estimation algorithm starts to fail com¬ 
pletely. Nevertheless, it is quite promising that even for L p ii ot = 
500, Sp b is lower than 2% for all WNR > — 5 dB. 

C. Estimation Based on SS Pilot Sequences 

So far, an estimation of the SCS receiver parameter A r based 
on a known SCS watermark has been proposed. However, it is 
also possible to estimate the scale factor g, and thus, A r = gA , 
with help of an additive SS pilot watermark. Here, we present 
an analysis of the estimation accuracy Sa t > as defined in Sec¬ 
tion VI-B, when using SS pilot watermarks and compare the 
result with those for SCS pilot watermarks. 

We consider again the attack channel defined in (27). How¬ 
ever, now, we assume that w is a pseudo-noise sequence of 
length L w = i p ;i 0 t with zero mean (J2n=i * w n = 0) an d 
power cr 2 = (l/L p iiot) En=T w n- Throughout this analysis, 
an IID host signal x and additive noise signal v is assumed so 
that x„ = x and v n = v , respectively, w is known to the wa¬ 
termark receiver so that g can be estimated from r based on the 
correlation c between r and w, that is 

^ -Epilot 

c = - - V' r n w„. (37) 

Tpilot ^ 


The unbiased estimate g of g derived from c is derived as fol¬ 
lows: 


E {r n } = E {(?(x n + w n ) + v n } = gE {x} + E { v} + gw n 


E Vc} = - - £ E {r n } i 

Tpiiot ^ 


- -Epilot -Epilot 

- (^E {4 + E MM* + J — J2 W n 

>ilot ^ Tpilot Ml 


sE x tEH ^ fJ ^ 2 

- T - Wn +7 - M w n 

M lot Ml L P‘ lot M 


Epii°t M 

cri 


W n = 9 °l 


where (41) describes the estimation rule for g using the SS pilot 
watermark w that is known to the receiver. Next, the variance of 
g dependent on the pilot length E p ii 0 t is derived. For simplicity, 
we assume that the host signal x and the attack noise v are 
mean-free (E {x} = 0 and E { v} = 0) so that the variance of x 
and w is given by = E {x 2 } and <r 2 = E { v 2 }, respectively. 
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Fig. 20. Estimation performance for (left) L P ii ot = 1000 and (right) L P ii ot = 2000. The performance for SS pilot watermarks and SCS pilot watermarks is 
compared (DWR = 20 dB, Lbinfc = 3, Lbinr = 50). The experimental results are averaged over 1000 simulations. 


The derivation of the variance Var {g\ is tedious but not difficult 
so that only the main steps are presented here: 


e {i 2 }=( 9 4) 2 + £ ^ ± ^4 

-C/pilot 

Var{c} =E{c 2 } -E{c} 2 = a 2 


Var {g} = Var < — } = 


ipilot 

Var {c} _ g 2 a 2 x + (J 2 v 


wir- 

ul 


Epilot^iv 


g^/al + a 2 /^ 

ipilot 


(42) 

(43) 


(44) 


We observe that Var \'g\ depends on the WNR via a 2 /a 2 , and 
on the DWR via a 2 /< 7 ^. The term <J 2 /<Jw dominates for realistic 
DWRs about 20 dB and WNR > —10 dB. Further, we observe 
that Var {g} decreases with increasing pilot length Z p ii ot . 

Fig. 20 compares the achieved estimation accuracy using SS 
pilot watermarks and SCS pilot watermarks for E p ii 0 t = 1000 
and Epiiot = 2000. Note that the estimation accuracy S,\ r for SS 
pilot watermarks can be computed theoretically from Var {g} 
via 


^E{(A r 5 A) 2 } y E{( £ A _ 5A)2} 

^ = - ~gA - = -- 

= y/E {(g-g) 2 } = yVar {g} (45) 

a a 

The results shown in Fig. 20 clearly demonstrate the superiority 
of the estimation algorithm based on SCS pilot watermarks. The 
advantage of the SCS pilot watermarks stems from the reduced 
influence of host-signal interference on the estimation accuracy. 


VII. Inverse SCS 

In some applications, it is desired to recover the original 
signal from the watermarked signal after watermark reception. 
Examples are information hiding applications dealing with 
medical images [38] or multiple watermark reception. In 
applications dealing with medical images, the goal is mainly 
to recover the original signal with a minimum amount of 


distortion. In multiple watermark reception, the interference of 
the first decoded watermark on other embedded watermarks 
should be minimized. For this, the already decoded watermark 
is exploited to remove the corresponding embedding distortion 
as much as possible. 

Perfect recovery of the original signal might be impossible 
in many practical cases, e.g., attack noise cannot be removed 
in general. However, in some cases, it is sufficient to produce 
a signal that is closer to the original signal than the received 
signal. In this section, ways to invert SCS watermarking are dis¬ 
cussed. In practice, the receiver sees an attacked watermarked 
signal. Here, a simple AWGN attack is considered again. For 
completeness, the noiseless case is discussed first. Throughout 
the section, it is assumed that the transmitted sequence of wa¬ 
termark letters d and the correct key sequence k are perfectly 
known, e.g., correct decoding has been performed, which can 
be treated without loss of generality as d = 0 and k = 0. The 
effect of possible remaining bit errors after error correction de¬ 
coding, and thus imperfect knowledge of d, is not investigated. 
However, it is obvious that for low BERs, the influence of the 
incorrect inverse mapping applied to those samples with incor¬ 
rectly estimated dither samples d n on the overall quality im¬ 
provement by inverse SCS is negligible. 

A. Inverse SCS in the Noiseless Case 

In the noiseless case, the watermark decoder receives 
the signal r = s. In this case, the deterministic embedding 
procedure can be inverted perfectly. With y n from (12), the 
host signal x n can be reconstructed with the next valid SCS 
codebook entry 

r q , n = Qa jfVi - A + k n j + A + k 7 ^j (46) 
by 

Xn = r q n Vn . (47) 

1 — a 

The perfect invertibility of SCS is also illustrated by the 
input-output characteristic of SCS embedding for a = 0.6, 
d n = 0, and k n = 0, which is shown in Fig. 3. The input-output 
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characteristic of SCS embedding is a strictly increasing func¬ 
tion so that the inverse mapping in the noiseless case exists. 
This inverse mapping is obtained by mirroring the input-output 
characteristic of SCS embedding at that for the identity map¬ 
ping Xn = s n . 


B. Inverse SCS After AWGN Attack 

Inversion of SCS watermarking after transmission over an 
AWGN channel is considered. Contrary to the noiseless case, 
it is impossible to reconstruct s from the received signal r, even 
with perfect knowledge of d because the transmitted value s n 
also depends on the original signal value x n that is not known to 
the receiver. Consequently, it is impossible to recover the host 
signal perfectly; however, one can at least try to find an estimate 
x so that for the distortion holds D (x, x) < D (x, r), where 
the MSE distortion measure is adopted. In the following, it is 
assumed that the channel noise cr 2 is smaller than or equal to 
the noise variance crp design for which the SCS watermark has 
been designed. 

1) Estimation of the Original Signal: The minimum mean- 
squared error (MMSE) estimate x n of the original signal sample 
x n should be derived for each received sample r n . IID signals 
are assumed so that the sample index n is suppressed in the fol¬ 
lowing. With help of the known key sequence sample k and 
known watermark letter d , the deviation y £ [—A/2, A/2) 
from the next valid SCS codebook entry r q is given by 

V — r q — r (48) 

For AWGN attacks, the most likely corresponding quantized 
original signal sample is r q . Thus, the MMSE estimate x is 

x(r q , y) = argnrinEp^fej {(% - x) 2 } 

x t eR 

= r q - argmin ^ Px ( x \ r ) {('0, ~ qt ~ x) 2 } 
q t e[- A/2, A/2) 

— r q — q(r q , y) (49) 

where r q is no longer considered within the minimization, and 
q( r q, y) £ [—A/2, A/2) has to be chosen such that the MSE 
E m0cW {(x-x) 2 } is minimized. Straightforward analysis 
shows that q(r q , y) has to be computed by 


q(r qi y) = E Mx \ r) {x}-r q 

/ OO 

xp x (x\r= r q -y) dx — r q . (50) 

■OO 


Thus, to solve the estimation problem, the conditional 
PDF p x (x\r= r q — y) must be known. It is assumed that 
p x (x\r=r q — y) is independent from r q , which is approx¬ 
imately valid for AWGN attacks and an almost flat PDF 
p x (x) in the range of one quantization interval, e.g., fine 
quantization, so that q(r q , y) = q(y ). Thus, the random 
variable y with support in [—A/2, A/2) is introduced, and the 
PDF p x (x\r = r q — y) = p x {x\y= y) is considered in the 
following. 

First, Bayes’ rule is applied, which yields 

, , x Px{x)p y (y=y\x=x) 
ft(l|y=!,) = —— 

for — oo < x < oo. (51) 



y,q 

Fig. 21. PDFs of received extracted data before and after inverse SCS mapping 
(WNR = 0 dB). 

Next, Py (y = y\x = x) has to be computed. Due to the quan¬ 
tization involved in the embedding procedure, an elegant 
closed-form of p y (y=y\x=x) does not exist. Another 
difficulty is the unlimited support of the random variable x. 
However, it turns out that for a 2 < design and r q = 0, a 
sufficiently accurate approximation is obtained by considering 
only x £ [-3A/2, 3A/2). 

With this approximation, the assumption of white Gaussian 
attack noise v of power cr 2 and the numerical representation of 
Py {y\d)i (51) can be evaluated, leading to a numerical represen¬ 
tation of p x (x\y = y). Applying these results to (50) yields the 
desired estimate q(y). For illustration purposes. Fig. 21 depicts 
the PDFs of y, and the resulting q for WNR = WNR des ign = 0 
dB. 

2) Achievable Distortion Reduction: Finally, the achieved 
distortion improvement is investigated. The improvement 
is measured in terms of the difference between the docu- 
ment-to-attack power ratio (DAR) before (DAR,,) and after 
(DAR*) the mapping, which is given by 

DAR imp = DAR ,. - DAR,, = 10 log 10 ° j X ' r) dB. (52) 

L) (X, X) 

Unfortunately, the result for WNR = WNR des ign is rather 
disappointing with a maximal distortion improvement of 
DARj mPi max = 0.04 dB. This value has been obtained via 
simulations and numerical evaluation of ~E px ( x \ r } {(i — x) 2 }. 
Obviously, the optimal quantizer step size in SCS is such that 
after AWGN attacks, the watermark embedding distortion is no 
longer invertible. Fig. 22 depicts the same result for the case 
of the SCS quantizer step-size design for a noise power being 
6 dB above the given channel noise power. In this case, the 
maximum distortion improvement is about 2.2 dB. Although 
this improvement might be of interest in practice, it is important 
to emphasize that such an improvement could be obtained only 
for very mild channel conditions. Note that the dependency 
of DAR imp on the WNR is due to the variable choice of A, 
depending on the WNR. 

It has to be concluded that the inversion of SCS watermarking 
after AWGN attacks is practically impossible or at least ineffi¬ 
cient. Nevertheless, the derived inverse SCS mapping might be 
useful. Suppose that the owner of a signal stores only the SCS 
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Fig. 22. Distortion improvement for inverse SCS mapping after AWGN attack 

wi* <4 = ^, design /-i- 

watermarked version and erases the original. In this case, the 
SCS watermark might be designed for strong robustness, that is, 
low WNRs. However, even without an explicit attack, the wa¬ 
termarked signal is slightly distorted due to quantization, which 
might occur when storing the data. This quantization can be ap¬ 
proximated by low-power noise. In such a scenario, the inverse 
scaling derived for the noiseless case might be not appropriate, 
but the MMSE estimation removes a good deal of the distortion 
introduced by the SCS watermark, as demonstrated in Fig. 22. 

VIII. Conclusions and Outlook 

Information embedding into IID original data and an attack 
by AWGN has been investigated. The decoder has no access to 
the original data. This scenario can be considered to be commu¬ 
nication with side information at the encoder for which a the¬ 
oretical communication scheme has been derived by Costa in 
1983. In this paper, a suboptimal practical version of Costa’s 
scheme has been studied. The new scheme is named “scalar 
Costa scheme” (SCS) due to the involved scalar quantization 
during encoding and decoding. A performance comparison of 
different blind watermarking schemes shows that SCS outper¬ 
forms the related DM techniques for low WNRs and performs 
significantly better than state-of-the-art blind SS watermarking 
for the relevant range of WNRs. The latter result is mainly due to 
the independence of SCS from the characteristics of the original 
signal. SCS combined with coded modulation achieves a rate of 
1 bit/element at WNR > 8.3 dB, which is within 1.6 dB of the 
SCS capacity. For WNR ss 1.5 dB, SCS communication with 
rate 1/3 turbo coding achieves BER < 10 -5 . For lower WNRs, 
SCS should be combined with the spread-transform (ST) tech¬ 
nique so that SCS operates effectively at a WNR « 1.5 dB. 

Two further topics that are relevant for the usage of SCS in 
practical information hiding systems are investigated. These 
are the robustness to amplitude scaling on the watermark 
channel and the removal of watermark embedding distortion by 
authorized parties. Robustness against amplitude scaling can be 
achieved via robust estimation of the proper SCS quantizer step 
size at the receiver, as described in Section VI. In Section VII, it 
is shown that the reduction of watermark embedding distortion 
is possible for low attack noise. 


The performance gap between SCS and ICS has to be bridged 
by constructing more complicated codebooks and by extending 
the embedding and detection rule to nonscalar operations. Re¬ 
search in this direction has been started, e.g., by Chou et al. 
[18], However, SCS might still remain an attractive technique 
for many information embedding applications due to its simple 
structure and host signal independent design. 
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